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In 2003, the Consortium completed most of the studies recommended by the 
research agenda established in 1996, including the large Annenberg Research 
Project. Much of the research focused on the implementation of the policy to 
end social promotion and its consequences, the improvement of teaching and 
learning, the state of Chicago’s public high schools, and educational technol- 
ogy. In addition, surveys were administered three times, and schools received 
their own results organized around their progress with respect to the Essential 
Supports for Student Learning. Since 1997, the Consortium has published 40 
reports. (For a complete list, see www.consortium-chicago.org/publications,) 

As we were coming to the end of our study cycle, the Steering Committee 
held several discussions regarding priorities for future research. This document 
sets forth a research agenda that continues some Consortium studies and out- 
lines new investigations. Guiding this agenda are the concerns of the Steering 
Committee, major policy and program initiatives of the Chicago Public 
Schools (see An Education Plan for the Chicago Public Schools),^ and capacity 
of the Consortium’s research team. To augment our research capacity, we 
will continue to seek the participation of external scholars who bring spe- 
cialized expertise. 

This research agenda aims to provide results that are directly useful to CPS 
leadership, local schools, and the education reform community; to bring evi- 
dence to bear on the theory of action behind major initiatives; and to supply 
evidence that is relevant to long-term policy issues. 

We have listed more studies than we can complete during the next five years. 
At this point, we are unsure of which of the larger ones will be funded, and 
when we can engage outside scholars. Including these studies in the agenda 
indicates the Consortium’s priority areas of concern. See Tables I and II (pages 
15-16) for a list of all studies and the proposed time lines. 
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Student Outcomes and School Productivity 

These studies aim to answer one of the most fundamental questions: whether 
over time Chicago students are making greater learning gains and raising their 
level of achievement. 

ACADEMIC PRODUCTIVITY 

Stuart Luppescu has initiated analyses of the changes in academic productivity 
in CPS elementary schools during Phase II of Chicago school reform. This 
parallels our earlier work on productivity in Phase I.^ The goal of this research 
is to examine the degree to which Chicago’s elementary schools improved over 
the last eight years in terms of students’ learning in reading and mathematics. 
Using Iowa Tests of Basic Skills (ITBS) data, the productivity indicator esti- 
mates each school’s contribution, or value-added, to the learning of students 
enrolled at that school. In addition, we intend to examine how these develop- 
ments are distributed across different kinds of students and schools. For ex- 
ample, in our Phase I analyses, we found a disturbing pattern of weak 
improvements in a subset of very disadvantaged, racially isolated African- 
American schools. Many of these schools were the explicit targets of Phase II 
reforms. Did they fare any better during the later period? Similarly, what 
kind of progress did schools with high proportions of English-language 
learners make? 

Beyond these substantive concerns, this research will also break new 
ground methodologically. Since 2000, Consortium Senior Director Anthony 
Bryk has been collaborating with Professor Steve Raudenbush (University of 
Michigan) on developing a more refined statistical model for estimating trends 
in school and classroom effects from students’ longitudinal test-score data. The 
model assumes that each student has an academic growth trajectory that is 
shaped jointly by the student’s latent growth parameters and by exposure to 
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instruction in a given class and school. Formally, the model involves repeated 
measures of students over time within different classrooms and schools. In 
addition, it permits estimation of growth curves for each student and time 
trends for classroom and school effects. 

One logical use of the new productivity indicator would be to compare 
it against the benchmarks for measuring adequate yearly progress that have 
been established by CPS and the state under the requirements of No Child Left 
Behind (NCLB). It is possible that under NCLB a school could be judged as not 
making adequate yearly progress, yet the more technically valid productivity 
measure would show improvement. The reverse is also possible. We may want 
to examine such differences. 



ILLINOIS STANDARDS ACHIEVEMENT TESTS (ISAT) 

We analyzed the ITBS extensively because it has been the main stable testing 
program in Chicago for decades, and it played a significant role in accountabil- 
ity. One strand of the analyses led us to probe the technical weaknesses of 
the test, and from time to time we have offered recommendations for its 
improvement. 

The significance of the ISAT has been growing and will especially be impor- 
tant under the demands of NCLB. We have begun to undertake in-depth 
technical analyses of the test as we have done with the ITBS. In February of 
this year, we released the first report with ISAT results: Research Data Brief: 
How Do They Compare? ITBS and ISA T Reading and Mathematics in Chicago 
Public Schools, 1999 to 2002. We intend to continue to monitor the ISAT results. 

HIGH SCHOOL TESTS 

For high schools, Steve Ponisciak has begun analysis of the Prairie State 
Achievement Examination (PSAE) data. Eleventh graders take the test, which 
consists of writing, science, and social science assessments developed by the 
Illinois State Board of Education; the ACT Assessment, which includes reading, 
English, mathematics, and science reasoning; and two Work Keys assessments— 
Reading for Information and Applied Mathematics. Our goal would be to 
establish periodic indicators that would allow us to summarize progress in 
student learning over time. It is possible to produce a value-added indicator by 
controlling for students’ basic skills level at the end of eighth grade and for the 
specific elementary schools from which they graduated. 
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The Steering Committee recommended that the Consortium advocate for 
high-quality assessment. Based on our study, Academic Productivity of Chicago 
Public Elementary Schools,^ we will submit recommendations to the Illinois 
State Board of Education regarding development of future tests. In addition, the 
Consortium will stay informed of the work of the local Commission on Im- 
proving Curriculum-Based Assessment that is being led by Don Stewart, Chi- 
cago Community Trust, and Sam Meisels, the Erikson Institute. 

NO CHILD LEFT BEHIND ACT OF 2001 (NCLB) 

The law has mandated dramatic expansion of state, and therefore, local account- 
ability systems, and its impact has begun to take hold in the Chicago Public 
Schools. It requires states to establish challenging standards in reading and 
mathematics, annual testing for all students in grades three through eight, and 
annual statewide progress objectives ensuring that all groups of students reach 
proficiency within 12 years. Assessment results and objectives must be disaggre- 
gated by poverty, race, ethnicity, disability, and limited-English proficiency to 
ensure that no group is left behind. School districts and schools that fail to 
make adequate yearly progress (AYP) toward statewide proficiency goals will, 
over time, be subject to improvement, corrective action, and restructuring 
measures aimed at getting them back on course. Schools that meet or exceed 
AYP objectives or close achievement gaps will be eligible for State Academic 
Achievement Awards. 

The school system must provide students in schools that do not meet 
the annual progress requirements the opportunity to attend better public 
schools and the necessary transportation. For students attending schools that 
have failed to meet state standards for at least three of the four preceding years, 
schools must permit low-income students to use Title I funds to obtain supple- 
mental educational services. If schools fail to make AYP for five years, they run the 
risk of reconstitution. Unfortunately, the Bush administration has withdrawn 
support for the original level of funding, and the Congress has appropriated 
only a fraction of the money originally intended for NCLB. 

So far, the main impact on Chicago has been to give students the oppor- 
tunity to transfer and to supply supplemental services. During 2002-03, about 
29,000 students were eligible to request a transfer; 2,300 students applied, and 
about 1,200 transferred to other schools. In 2003-04, 270,000 students, or over 
50 percent, were eligible to transfer. About 19,000 requested transfers, but space 
was available for only about 1,000 students. In addition, it is evident that the 
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classification of schools will be unstable. Forty schools that could receive 
transfers in the first year did not meet second-year standards and will have to 
give their students the opportunity to transfer out. In subsequent years, the 
consequences of the law are potentially even more drastic. 

While the Consortium does not recommend a specific study at this time, 
we have noted in other sections of this agenda the need to pay attention to the 
impact of the new law (see sections on academic productivity and mobility). 

The local consequences of NCLB will continue to unfold, and the Consortium 
intends to monitor the situation and if necessary conduct appropriate analyses 
to inform local (and perhaps national) policy. It is possible that this work could 
draw resources away from other topics in this agenda. 
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Deeper Examination of the Essential Supports 

ORGANIZING SCHOOLS FOR IMPROVEMENT 

We will finish the book on the essential supports for student learning. The 
book examines elementary school development from 1990-1996. It sets forth a 
theory of essential supports and community contexts that influence the im- 
provement of student learning. Extensive empirical data validate and extend the 
theory, and two case studies illustrate ways the essential supports operate and 
mutually support one another. The book examines the essential supports across 
different kinds of school communities, drawing attention to the unique needs 
of schools in impoverished and dangerous neighborhoods that have few com- 
munity resources other than the school. We also plan to add a postscript about 
how all the improving schools fared in the subsequent five-year period. We 
have written five of eight chapters. The University of Chicago Press has invited 
us to submit the manuscript. 

SCHOOL LEADERSHIP: PRINCIPALS 

We know from past studies that school principals play a powerful role in 
facilitating school development. Mark Smylie, Al Bennett, Holly Hart, and 
David Schalliol have begun to analyze principal and teacher survey data from 
multiple years to describe the characteristics of the CPS principal workforce 
and the support principals received to do their work. The last study of princi- 
pal leadership occurred over 10 years ago. This study would provide the school 
system, those in the city concerned with school leadership development, and 
the public generally with important information concerning who Chicago’s 
principals are, the demands of their work, how they are perceived by teachers, 
and what needs they may have for development. In addition, it will focus on 
their relationships with local school councils (LSCs) and with the new Area 
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Instructional Officers. The 2003 principal and teacher questionnaires contain 
items on LSCs that had been asked in surveys in the early and mid-1990s. 

Particular attention will be paid to understanding the differences in principal 
practice between high- and low-performing schools, and differences in leader- 
ship practice between schools in the lowest economic quartile and the rest of 
the school system. 

PROFESSIONAL CAPACITY: HUMAN RESOURCE DEVELOPMENT 

The Education Plan announced initiatives to build professional capacity 
through new hiring and ongoing professional development of existing staff. 
These initiatives focus on teacher recruitment, certification, mentoring of new 
teachers, introduction of a career ladder, centrally-supported professional 
development programs, and extra supports for high-need schools in attracting 
and retaining high-quality candidates. As the plan was being created, the Chi- 
cago Public Education Fund commissioned a study that identified $123 million 
in the CPS budget earmarked for professional development. The study con- 
cluded that professional development spending was not integrated into a com- 
prehensive strategy for improving instruction.'^ CPS leadership accepted the 
results and has begun to harness these budget allocations to implement the 
objectives of the Education Plan. 



Currently there is little systematic data available on the city’s teaching force. 
The Consortium contemplates a set of integrated studies that would address 
this need. Key indicators need to be developed in five areas of human resource 
development: teacher training and experience; recruitment and induction; 
movement of teachers from one school to another within the system and to 
other systems; support for teachers within their schools; and ongoing, high- 
quality professional development. Equity analyses of the distribution of quali- 
fied teachers across different kinds of schools are vitally important.^ These 
would include small schools, charter schools, and schools serving diverse 
student populations with respect to race and ethnicity and economic circumstances. 

Steering committee members expressed considerable support and offered 
many suggestions for this topic. They noted the need to recognize the develop- 
mental stages teachers go through in learning their craft. In addition, they drew 
attention to the new requirements for certification and how these will unfold. 
Others raised questions about financial and nonfinancial incentives that help to 
keep good teachers. 
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Arne Duncan has instituted a policy to close nonperforming schools and to 
encourage the creation of new schools, including charter, contract, and small 
schools. Hence, studies in this area should address how new faculties are consti- 
tuted, the degree to which new schools succeed in attracting qualified teachers, 
and whether struggling schools lose qualified teachers to these or other schools. 

The 2003 teacher and principal surveys asked questions pertinent to human 
resource development. We continued, for example, our teacher background and 
professional development items. There is a new item on the teacher question- 
naire asking whether the teacher had participated in one of several different 
alternative certification programs. In addition, the Chicago Public Education 
Fund financed a page of questions for principals about teachers in their school 
who are seeking or have obtained certification from the National Board for 
Professional Teaching Standards. 

This is an integrated series of studies for which it would be advantageous to 
recruit an outside scholar to collaborate with a local director to lead the re- 
search activities. It will be necessary to raise significant funds. 

INSTRUCTION AND ITS IMPROVEMENT 

A major focus of the CPS leadership is on instruction and its improvement, and 
Anthony Bryk will lead an integrated program of research in this domain. It 
requires building a database at the classroom level that links students and 
teachers. Through the Annenberg Research Project, we were able to collect 
such data for about 60 classrooms, consisting of both reading and mathematics 
instruction for grades three, six, and eight. In these classrooms, teachers com- 
pleted the regular Consortium teacher survey and a special survey on instruc- 
tion. In addition, field staff observed teachers and collected teacher assignments 
and student work. Finally, we have students’ full test score history in reading 
and mathematics. Hence, while the size of the sample is fairly small, we will 
begin to analyze the mix of content, academic demand, and pedagogical strate- 
gies teachers brought to the classroom and the relationship between classroom 
work and students’ learning gains. Appropriately, the Annenberg Research 
Project helped lay the groundwork for a more ambitious investigation of 
instruction and its improvement, and this analysis will serve as a bridge to the 
larger research agenda on instruction. 

To link teacher and student data, the 2003 surveys asked teachers to indicate 
their room number. In addition, with the help of a pilot test done in 2002 
by the Center for School Improvement, the Consortium expanded survey 
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measures to address new areas of instruction, the availability of materials and 
resources, the type and frequency of assessment, professional development, and 
the work of the reading coordinators that the central administration assigned to 
schools in the last two years. 

To expand classroom-level analysis to the entire system depends on a signifi- 
cant change in the CPS information management system to allow for the 
linking of teacher and student records. We expect this to happen in the next 
couple of years. By that time, the Consortium will have undertaken the first 
two sets of analyses and will be prepared to expand the investigation of instruc- 
tion and its improvement to all elementary schools. The Consortium also 
anticipates exploration of principal questionnaire data about their working 
relationships with Area Reading Coordinators and Area Instructional Officers. 
In addition, we may make greater use of the ISAT and launch further fieldwork 
to collect teacher assignments and student work (additional funds necessary). 
We would ultimately like to link teaching practice, teacher preparation and 
certification, and teacher recruitment and retention. 

Steering Committee members recommended that we also address the follow- 
ing questions: What is the importance of subject matter expertise? Is there 
coherence at the system level and school level between standards, curriculum, 
and assessment? What is the impact of class size and student discipline? What 
are the salient issues of race and ethnicity, culture, and language in improving 
instruction? 
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A Closer Look at Students in CPS 

CHILDREN AND THEIR ENVIRONMENTS 

We have begun to update our census indicators to produce a new report based 
on the 2000 census. The report, which will be available online, will produce 
citywide and community-area statistics that include the number of preschool- 
age children, the median family income, the count of school-age children who 
are non-English speaking, the households in linguistic isolation (no individuals 
14 and older who speak English well), the number of school-age children, the 
racial and ethnic composition of school-age children, the person(s) with whom 
children live, the types of schools that children attend (magnet, neighborhood, 
etc.), and the percent of children living below the poverty threshold. For each 
variable, the report will produce count, percent, and change in percent. For 
example, consider preschool-age children in community-area three. Readers 
will be able to select the count, percent, and change in percent of preschool-age 
children in the community area alone for 1990 and 2000. 

STUDENT MOBILITY 

In the last couple of years, as a result of federal and local policies, student 
mobility has become a more salient issue. First, NCFB has forced school sys- 
tems to allow students to transfer to other schools if their school has not im- 
proved in three years. While overcrowding of some Chicago schools has 
limited parents’ options in this regard, in general the policy is likely to increase 
mobility. Second, the CEO of the school system has instituted a policy of 
school closures combined with the opening of new charter, contract, and small 
schools, which will also stimulate student mobility. The closing and demolition 
of Chicago Housing Authority buildings has also sent families to find other 
housing and schools. Finally, there are many parents who seek to improve their 
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children’s opportunities by either moving to a different neighborhood or 
requesting a transfer to a different school. 

Previous research has shown that school mobility, especially if it occurs 
often and within school years, can be harmful. Schools with high percentages of 
mobile students tend to see the pacing of instruction slow down as teachers 
work to accommodate new students.^ Hence, for this reason, as well as to 
inform statistical analyses for other studies in this agenda, the Consortium 
intends to produce a data brief on this topic. 

CHILDREN AND YOUTH IN CHALLENGING CIRCUMSTANCES 

During the past year, the Consortium has collaborated with Chapin Hall on a 
planning study that provides descriptive information about children who have 
been abused and neglected, and/ or are in foster care. This information was 
compiled for the CPS leadership (and that of other institutions) as it determines 
future investments in this area. A main finding was the importance of close 
monitoring of children’s school experiences in the first year of entering foster 
care, when the child needs considerable support, and the decisions about place- 
ment in programs or schools must be carefully considered. In addition, despite 
concerns about labeling and other issues, there is a great need for communica- 
tion between the child’s social worker, foster care family, and classroom 
teacher. The transition to high school was also identified as a critical event 
requiring extensive communication, understanding, and judgment. 

Based on this preliminary analysis, we plan to develop a proposal in collabo- 
ration with Chapin Hall and other partners to undertake further work in this 
area, including analysis of CPS students who have been involved in the juvenile 
justice system. 

ENGLISH-LANGUAGE LEARNERS 

The Steering Committee expressed considerable interest in the quality of 
educational experiences for English-language learners. We understand that the 
CPS Planning Office intends to undertake an internal analysis of bilingual 
education. This activity will be similar to the preliminary analyses they com- 
pleted of CPS graduates’ postsecondary education and training and of students 
in foster care. Planning staff will compile existing data and conduct interviews 
and focus groups. As we have done with the previous two CPS planning stud- 
ies, the Consortium will stay abreast of this project and make a determination 
of whether we should and could undertake a study in this area. 
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IV. 



Follow-Up on Previous Research 

HIGH SCHOOLS 

Goal 6 of the Education Plan calls for strengthening existing high school pro- 
grams, and the Steering Committee recommended that the Consortium give 
much more attention to research in this area. They reasoned that in our 13 
years of experience, we have learned a great deal about the mechanisms for 
improving elementary schools. However, there is much less certainty about 
how to reform high schools. 

With encouragement from the Steering Committee, we have decided to 
undertake two major studies with respect to high schools. First, Elaine 
Allensworth will lead a comprehensive analysis of dropout trends, the process 
of dropping out, and the school and program factors that appear to influence 
students’ decisions to stay enrolled or leave school. Consortium estimates show 
that dropout rates have remained alarmingly high. Over 40 percent of students 
drop out by the time they reach 19 years of age.^ 

To follow up Allenworth’s previous research in this area, she plans to pro- 
duce a data brief that updates systemwide trends, analyzes factors behind those 
trends, and examines dropout rates by age, race and ethnicity, and gender (and 
combinations of these), and by elementary school attended, by high school, and 
by neighborhood. This will provide basic information on the phenomenon in 
CPS, as well as evidence regarding differences among schools with respect to 
keeping students enrolled. In addition, she intends to examine CPS drop-out 
patterns— who drops out and when they drop out. This includes charting the 
movement of dropouts in and out of the system, between schools, through 
alternative schools, and incarceration. 
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Allensworth also intends to probe further the characteristics of schools that 
succeed in holding students through graduation, including evidence of strength 
in the essential supports such as safety, peer relationships, and teacher-student 
trust. With additional funding, it may be possible also to conduct interviews 
with teachers, students, and administrators about the programs and policies 
they are implementing to prevent students from dropping out, including mak- 
ing accommodations in the curriculum, working with students who are chroni- 
cally truant, and working with parents. 

Second, Melissa Roderick has decided to undertake a complex study of CPS 
students who go on to postsecondary education and training. Initially this will 
involve obtaining national data sets of students in postsecondary institutions 
and matching CPS graduates. In addition, she plans to follow several hundred 
graduates over a period of three years. The purpose of the study is to map 
where CPS students go for postsecondary education, understand how they 
choose institutions, examine their preparation for the academic and social 
demands they confront, and evaluate their persistence. Ultimately, this study 
should inform the development and improvement of CPS high schools. 

With respect to high school reform efforts, already we are providing assis- 
tance to the evaluation of the Fry Foundation’s high school leadership reform 
project. We are analyzing quantitative indicators for the Institute of Metropoli- 
tan Affairs at Roosevelt University, where James Lewis, a member of our 
Steering Committee, is a coprincipal evaluator along with Al Bennett, a direc- 
tor of the Consortium. We also obtained funding from the Spencer Foundation 
to study the Bill and Melinda Gates’ Foundation-supported small school efforts 
in Chicago. Research analysts Sue Sporte and Macarena Correa have completed 
the first round of fieldwork. 

EDUCATIONAL TECHNOLOGY 

We will continue to monitor access to and professional development on tech- 
nology. Growth in these activities would yield greater use of technology among 
teachers and students. 
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Assisted Studies 



We are providing data and assisting several groups, including members of the 
Consortium Steering Committee. The largest assisted study is longitudinal 
surveys of teachers and students in Catholic Elementary Schools. The principal 
investigator is Maureen Hallinan, Notre Dame University. 




Table I: Research Agenda by Topic, FY 2004 - 2008 
Consortium on Chicago School Research 



15 



Consortium on Chicago School Research 




Depends on success in raising funds. ** Assume these will constitute one-quarter of budget. 9/03 



Table II: Research Agenda by Study Type, FY 2004 - 2008 
Consortium on Chicago School Research 
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Depends on success in raising funds. 9/03 
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